1. Field of the Invention
The present invention relates to a MDS multi-layer neural network and its designing method, and more particularly to a multi-layer neural network and the designing method thereof which can easily realize VLSI circuitry for a multi-layer neural network.
2. Description of the Related Art
Recently, in the pattern recognition field, neural networks which can do large-scale parallel processing in real-time have been introduced. In 1988, Hans P. Graf et al. of Bell laboratories, presented a pattern recognizing neural network wherein amplifiers, having their inputs and outputs interconnected through a resistive connecting device in a matrix, constitute synapses and neurons of the neural network, and a pair of switches which switch according to the data in a pair of RAM cells, constitute neuron interconnections.
The present inventor has filed numerous applications including: an adder (U.S. application Ser. No. 07/473,653), a multiplier (07/473,633), an A/D converter (07/473,631, 07/473,634), a pattern classifier (07/473,464), etc. These related applications disclose a neural network having synapses constructed using PMOS and/or NMOS transistors, and including neurons made from buffers in the form of cascaded pairs of CMOS inverters. The aforementioned neural circuits are based on a one-layer neural network model and solve only linearly separable problems, but not linearly unseparable problems (non-linear problems). Accordingly, their application is extremely limited. For instance, an XOR circuit cannot be properly modelled by a one-layer neural network. The limitations of one-layer neural networks are overcome by properly designed multi-layer neural networks. It is known that a neural network can learn by a back-propagation algorithm (refer to IEEE ASSP MAGAZINE, pages 4 to 22, April, 1987).
There are more restrictions, however, in the hardware implementation of multi-layer neural networks than in software simulated algorithms in a computer. This is due to the fact that hardware implementation of a multi-layer neural network depends on current VLSI technology. As a result, circuit implementation of connection weight values for neural circuits and non-linear functions is not as readily attainable as with software simulated algorithms. Although real number calculations with floating point operation and also neural network models requiring increased interconnection capability can be achieved using software, such software implemented operations become difficult with VLSI technology.